Control device and imaging apparatus

ABSTRACT

A control device includes: a correction control unit configured to acquire a blur detection signal detected by a blur detection unit to calculate a correction amount of an image blur and control an image blur correction unit configured to correct the image blur; a subject detection unit configured to detect a position of a subject in a photographed image and acquire position information of the subject in the photographed image; a tracking control unit configured to perform tracking control of the subject based on the position information of the subject acquired by the subject detection unit to; and a setting unit configured to set a control state of the tracking control unit. The setting unit sets the control state of the tracking control unit by selecting the set control state from a plurality of control states including a first state in which a tracking subject selectable mode is not set and a second state in which the mode is set and a tracking subject is not selected. The correction control unit acquires information on the control state selected by the setting unit and performs control to change characteristics of calculating the correction amount so that an image blur correction effect in the second state is higher than an image blur correction effect in the first state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 15/368,200,filed Dec. 2, 2016 the entire disclosure of which is hereby incorporatedby reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a control device and an imagingapparatus.

Description of the Related Art

An imaging apparatus having an image blur correction function ofsuppressing a blur of an image due to a hand shake or the like and aface detection function and a human detection function when a subject isa person is known. For example, a pattern for determining the face of aperson is determined in advance, and a portion which is identical to thepattern included in a photographed image is detected as a facial image.The detected facial image of the person is referenced in focus control,exposure control, and the like.

The following matters occur when photographing a moving subject orphotographing a subject in a telephoto mode with a large focal distance.When a subject moves to be positioned out of a photographic angle ofview, it takes a special skill of a photographer to track a continuouslymoving subject with high accuracy by only a manual operation only.Moreover, when photographing is performed using a camera having atelephoto lens, since the influence of a hand shake on an image blurincreases, it is difficult to maintain a main subject at the centralposition of a photographed image. When a photographer operates a camerato capture a subject within an angle of view again, an image blur iscorrected according to a hand shake amount occurring when thephotographer operates the camera intentionally. Due to this, dependingon the influence of image blur correction control, it may be difficultfor a photographer to perform fine adjustment to capture a subjectwithin an angle of view or position a subject image at the centralposition of a photographed image.

Japanese Patent Laid-Open No. 2010-93362 discloses an imaging apparatusthat automatically tracks a subject by moving a portion of an opticalsystem in a direction crossing an optical axis. The position of asubject is detected from an image signal acquired by an imaging elementand a subject tracking computation amount is calculated. The subjecttracking computation amount is combined with a blur correctioncomputation amount to perform subject tracking control while correctingan image blur.

SUMMARY OF THE INVENTION

A control device includes: a correction control unit configured toacquire a blur detection signal detected by a blur detection unit tocalculate a correction amount of an image blur and control an image blurcorrection unit configured to correct the image blur; a subjectdetection unit configured to detect a position of a subject in aphotographed image and acquire position information of the subject inthe photographed image; a tracking control unit configured to performtracking control of the subject based on the position information of thesubject acquired by the subject detection unit to; and a setting unitconfigured to set a control state of the tracking control unit. Thesetting unit sets the control state of the tracking control unit byselecting the set control state from a plurality of control statesincluding a first state in which a tracking subject selectable mode isnot set and a second state in which the tracking subject selectable modeis set and a tracking subject is not selected. The correction controlunit acquires information on the control state selected by the settingunit and performs control to change characteristics of calculating thecorrection amount so that an image blur correction effect in the secondstate is higher than an image blur correction effect in the first state.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an imaging apparatus according to afirst embodiment and a shaking direction of the imaging apparatus.

FIG. 2 is a diagram illustrating a configuration example of the imagingapparatus according to the first embodiment.

FIGS. 3A and 3B are diagrams for describing tracking control of adetected subject.

FIG. 4 is a functional block diagram of a tracking amount calculationunit according to the first embodiment.

FIGS. 5A to 5C are diagrams for describing a subject designation methodof the tracking control of the first embodiment.

FIG. 6 is a diagram for describing setting of a blur prevention cutofffrequency according to a subject position according to the firstembodiment.

FIG. 7 is a flowchart for describing the control according to the firstembodiment.

FIGS. 8A and 8B are diagrams schematically illustrating an imagingapparatus according to a second embodiment.

FIG. 9 is a diagram illustrating a configuration example of the imagingapparatus according to the second embodiment.

FIG. 10 is a diagram illustrating a configuration example of an imagingapparatus according to a third embodiment.

FIG. 11 is a diagram illustrating a configuration example of an imagingapparatus according to a fourth embodiment.

FIGS. 12A to 12H are diagrams illustrating the state of the imagingapparatus during tracking control according to the fourth embodiment andan example of a timing chart.

FIGS. 13A to 13H are diagrams illustrating an example of the state ofthe imaging apparatus during tracking control according to the fourthembodiment and an example of a timing chart.

FIGS. 14A to 14H are diagrams illustrating the state of the imagingapparatus during tracking control according to the fourth embodiment andan example of a timing chart.

FIGS. 15A to 15H are diagrams illustrating the state of the imagingapparatus during tracking control according to the fourth embodiment andan example of a timing chart.

FIGS. 16A to 16H are diagrams illustrating the state of the imagingapparatus during tracking control according to the fourth embodiment andan example of a timing chart.

FIG. 17 is a diagram illustrating a configuration example of the imagingapparatus according to the fourth embodiment.

FIG. 18 is a diagram illustrating a configuration example of the imagingapparatus according to the fourth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, respective embodiments of the present invention will bedescribed in detail with reference to the drawings. In each embodiment,an imaging apparatus including a control device having a function ofdetecting a shake to correct an image blur and a function ofautomatically tracking a subject will be illustrated. The presentinvention can be also applied to an imaging apparatus such as asurveillance camera, a web camera, or a mobile phone without beinglimited to a digital camera or a digital video camera. Moreover, thepresent invention can also be applied to lens-replacement andlens-integrated cameras.

First Embodiment

In a system that tracks a subject so that the position of a subjectimage is maintained at a specific position (for example, the centralposition) within a photographed image, subject tracking control iscontrol in which a camera performs correction automatically so that theposition of the subject image is at the specific position. Therefore, aphotographer does not need to perform framing to track the subjectduring tracking control. In this case, when a blur correction effect(hereinafter referred to as an image blur correction effect) is enhancedby broadening a frequency range of control characteristics whencorrecting an image blur, it is possible to perform photographing withsmooth subject tracking while eliminating the influence of a hand shakeor the like.

However, when such subject tracking is performed, although aphotographer can perform tracking by designating a subject in the screenin order to photograph a desired subject, it may be difficult todesignate the subject itself. For example, when the focal distance of aphotographing lens is very large (1000 mm or larger) , a subject imagemay move due to the influence of a hand shake and it may be difficult tocapture a subject within an angle of view to perform framing. In thiscase, when the image blur correction effect is enhanced by broadening afrequency range of the control characteristics of the image blurcorrection, it is possible to alleviate the difficulty of framing due tothe influence of a hand shake. However, when the image blur correctioneffect is enhanced, if the subject moves, image blur correction isperformed on a hand shake amount occurring when the photographeroperates the camera intentionally even when a photographer operates thecamera so that the subject falls within a photographic angle of view.Depending on the degree of the image blur correction effect, it may bedifficult to capture the subject within the angle of view again orfinely adjust the position of the subject image to be positioned at thecentral position of a photographed image. Therefore, in the presentembodiment, an imaging apparatus comprises a control device thatperforms image blur correction and subject tracking control and canimprove the easiness of the photographer's framing will be described.The control device is sometimes referred to as “an image positioncontrol device”, because the control device controls an image positionby performing image blur correction and subject tracking control.

FIG. 1 is a diagram for describing a shaking direction of an imagingapparatus according to respective embodiments of the present invention.A control device mounted on an imaging apparatus 101 includes an imageblur correction device and a subject tracking control device. FIG. 2illustrates a configuration of an imaging unit of the imaging apparatus101 and the functional blocks of an image blur correction process and anautomatic tracking process executed by a central processing unit (CPU)105 included in the imaging apparatus 101.

The image blur correction device performs image blur correction on anangular shake in pitch and yaw directions indicated by arrows 103 p and103 y in relation to an optical axis 102, for example. A Z-axisdirection of a three-dimensional orthogonal coordinate system is definedas an optical axis direction, a first axis orthogonal to the Z-axis isdefined as an X-axis, and a second axis orthogonal to the X-axis isdefined as a Y-axis. A direction around the X-axis, indicated by arrow103 p is the pitch direction and a direction around the Y-axis,indicated by the arrow 103 y is the yaw direction.

A release switch 104 for allowing a shutter to travel is provided in amain body of the imaging apparatus 101, and a switch opening/closingsignal is transmitted to the CPU 105 according to an operation on therelease switch 104. The release switch 104 is a two-step switch in whicha first switch (SW1) and a second switch (SW2) sequentially enters intothe ON state according to a pressing amount. The SW1 enters into the ONstate when the release switch 104 is half-pressed and the SW2 entersinto the ON state when the release switch 104 is fully pressed. When theSW1 enters into the ON state, focusing is adjusted by the driving of afocus lens and an appropriate exposure amount is set by the driving of adiaphragm. When the SW2 enters into the ON state, image data obtainedfrom an optical image exposed to an imaging element 106 is recorded on arecording medium.

The CPU 105 functions as the control device of the present embodiment.The present embodiment can be applied to an arbitrary optical apparatusincluding the CPU 105. The imaging element 106 and a blur correctionlens (hereinafter also referred to as a correction lens) 108 thatcorrects an image blur of a subject image on the screen by moving in adirection different from the optical axis 102 are positioned on theoptical axis 102 of the imaging optical system.

The imaging apparatus 101 includes an angular velocity detection unit(hereinafter referred to as an angular velocity meter) that detects anangular velocity of an angular shake. The angular velocity meter 103detects an angular shake in a rotation direction (the pitch direction)around the X-axis, indicated by arrow 103 p in FIG. 1 and a rotationdirection (the yaw direction) around the Y-axis, indicated by arrow 103y. A detection signal output by the angular velocity meter 103 is inputto the CPU 105, and a blur correction angle is calculated by a blurcorrection angle calculation unit 109. The output of the blur correctionangle calculation unit 109 is input to a sensitivity adjustment unit114.

When automatic subject tracking control is not performed, the blurcorrection angle calculation unit 108 calculates the blur correctionangle based on the angular velocity meter 103. Specifically, in the blurcorrection angle calculation unit 109, an offset subtractor 110 removesa direct current (DC) component added as detection noise from thedetection signal which is the output of the angular velocity meter 103to the angular velocity meter 103. Moreover, an integral filter unit 111executes an integration process and outputs an angle signal. The offsetsubtractor 110 removes the DC component using a high-pass filter (HPF),for example. Moreover, the integral filter unit 111 uses a filterexpressed by Expression (1) below. Expression (1) expresses a filter inwhich an integrator, the first term on the left side and a HPF, thesecond terminal on the left side are combined. This filter can beexpressed as an expression in which a time constant T is multiplied withan expression of a low-pass filter (LPF) having the time constant T.

$\begin{matrix}{{\frac{1}{s} \times \frac{Ts}{{Ts} + 1}} = \frac{T}{{Ts} + 1}} & (1)\end{matrix}$

The output of a blur prevention characteristics changing unit 112 isinput to the offset subtractor 110 and the integral filter unit 111. Theoutput of a panning determination unit 113 is input to the blurprevention characteristics changing unit 112. The output of a trackingswitch (SW) 121 is also input to the blur prevention characteristicschanging unit 112, the process of the blur prevention characteristicschanging unit 112 corresponding to the tracking SW 121 will be describedlater.

The panning determination unit 113 acquires the output of the angularvelocity meter 103 and the blur correction angle which is the output ofthe integral filter unit 111 and determines whether the imagingapparatus 101 is panning. Specifically, the panning determination unit113 compares the angular velocity of the imaging apparatus 101 detectedby the angular velocity meter 103 with a predetermined threshold. When apredetermined period (a determination period) has elapsed from the timepoint at which the detected angular velocity exceeded the threshold, itis determined that the imaging apparatus 101 is panning, and adetermination signal is output to the blur prevention characteristicschanging unit 112.

When it is determined that the imaging apparatus 101 is panning, thepanning determination unit 113 issues an instruction to the blurprevention characteristics changing unit 112 to increase a thrust towardthe center of a control range so that the blur correction angle outputby the integral filter unit 111 does not become too large. The thrustrefers to a control action to control the blur correction angle toapproach the center of a control range. Moreover, when it is determinedthat the imaging apparatus 101 is not panning, the panning determinationunit 113 issues an instruction to the blur prevention characteristicschanging unit 112 to decrease the thrust toward the center of thecontrol range . Here, the thrust toward the center of the control rangemay be changed gradually according to the magnitude of panning. Themagnitude of panning is the magnitude of the detected angular velocityor the length of a period in which the angular velocity exceeds thethreshold. In the present embodiment, a panning operation will bedescribed by way of an example. Since the same process is performed fora tilting operation except for the difference in direction, the detaileddescription thereof will not be provided.

The blur prevention characteristics changing unit 112 changes the blurprevention characteristics (image blur correction characteristics)according to a panning determination result and an instruction on themagnitude of the thrust toward the center of the control rangecorresponding to the blur correction angle. For example, when the blurcorrection angle which is the output of the integral filter unit 111 islarger than the predetermined threshold, the blur preventioncharacteristics changing unit 112 changes the blur preventioncharacteristics to control the blur correction angle to approach nearthe center of the control range. Control is performed according to themagnitude of the blur correction angle so that the blur correction angleapproaches near the center gradually. Specifically, control is performedso that the larger the blur correction angle, the larger the thrusttoward the center whereas the smaller the blur correction angle, thesmaller the thrust toward the center.

The blur prevention characteristics changing unit 112 changes the blurprevention characteristics by changing the frequency characteristics ofthe offset subtractor 110 or the integral filter unit 111. When aninstruction to decrease the thrust is received, the blur preventioncharacteristics changing unit 112 decreases the cutoff frequency of theHPF of the offset subtractor 110 and decreases the cutoff frequency ofthe integral filter unit 111. When an instruction to increase the thrustis received, the blur prevention characteristics changing unit 112increases the cutoff frequency of the HPF of the offset subtractor 110and increases the cutoff frequency of the integral filter unit 111.

As described above, the blur correction angle calculation unit 109performs a process of changing the thrust toward the center of thecontrol range according to the panning determination result and themagnitude of the blur correction angle. Due to this, it is possible tocalculate the blur correction angle while performing a panningdetermination process. The blur correction angle signal output by theblur correction angle calculation unit 109 is input to the sensitivityadjustment unit 114.

The sensitivity adjustment unit 114 amplifies the output of the blurcorrection angle calculation unit 109 based on zoom and focus positioninformation 108 and a focal distance and a photographing magnificationderived from these items of information and outputs a blur correctiontarget value. The zoom position information is acquired from a zoom unitand the focus position information is acquired from a focus unit. Thereason why the blur correction target value is calculated based on thezoom and focus position information 107 is because blur correctionsensitivity on an imaging plane in relation to blur correction stroke ofthe correction lens 108 changes depending on a change in opticalinformation such as zooming or focusing. The sensitivity adjustment unit114 outputs the blur correction target value to an adder 115.

When automatic subject tracking is not performed (the trackingcorrection amount which is the output of the tracking amount calculationunit 118 is 0), the adder 115 outputs the blur correction target valueobtained by the sensitivity adjustment unit 114. Only the output of thesensitivity adjustment unit 114 is input to a driving control unit 116.The correction lens 108 functions as a movable unit that shifts asubject position in a photographed image. The driving control unit 116controls the driving of the correction lens 108 to executes subjecttracking control and blur correction.

In the present embodiment, the driving control unit 116 executes imageblur correction (optical blur prevention) by driving the correction lens108 in a direction different from the optical axis direction. Althoughan image blur correction method which uses the correction lens 108 isemployed in the present embodiment, the present invention is not limitedthereto, and a method of correcting an image blur by moving an imagingelement within a plane vertical to the optical axis may be applied.Alternatively, electronic blur prevention which reduces the influence ofa shake by changing a slicing position of an image in each photographicframe output by an imaging element may be applied. Alternatively, aplurality of image blur correction methods may be combined. Furthermore,in the present embodiment, subject tracking control is performed bydriving the correction lens or the imaging element and changing theslicing position of an image using the optical blur prevention techniqueand the electronic blur prevention technique.

Next, a subject position detection process performed by a subjectdetection unit 117 will be described in detail. The imaging element 106acquires image information by photoelectrically converting lightreflected from a subject to an electrical signal. The image informationis converted to a digital image signal by an A/D conversion unit and thedigital image signal is transmitted to the subject detection unit 117.When a plurality of subject images are captured within a photographicangle of view, the following methods are used as a method ofautomatically recognizing a main subject from a plurality of subjects.

A first method of detecting a main subject detects a person. In thiscase, the subject detection unit 117 detects the face or a human body ofa subject. In a face detection process, a pattern for determining theface of a person is determined in advance, and a portion that matchesthe pattern included in a captured image can be detected as a facialimage of the person. The subject detection unit 117 calculates areliability indicating a probability as the face of a subject for eachof the detected subjects. The reliability is calculated from the size ofa face region in an image, the degree of matching with the face pattern,and the like, for example. That is, the subject detection unit 117functions as a reliability calculation unit that calculates thereliability of the subject based on the size of a subject in aphotographed image or the degree of matching between the subject and thesubject pattern stored in advance.

A second method of detecting a main subject uses a histogram of hue,saturation, and the like in a captured image. A process of dividing adistribution derived from the histogram of hue, saturation, and the likeof the image of the subject captured within the photographic angle ofview into a plurality of segments and classifying images captured in therespective segments is executed. For example, a histogram of a pluralityof color components is created for a captured image and is divided in amountain-shaped distribution range, images captured in regions belongingto a combination of the same segments are classified, and subject imageregions are recognized. By calculating evaluation values for respectiverecognized subject image regions, a subject image region having thehighest evaluation value can be determined as a main subject region.

After the main subject region is determined, it is possible to track themain subject region by detecting a region having a feature amountsimilar to the feature amount of the main subject region from imagessequentially captured by a subsequent live-view operation, continuousshooting, or video capturing using the feature amount of the mainsubject region. The feature amount of the main subject region iscalculated from a hue distribution, a hue size, or the like, forexample. The position information of the detected main subject is inputto the tracking amount calculation unit 118. The tracking amountcalculation unit 118 calculates a tracking correction amount so that thecentral position of the main subject image is positioned near the center(a target position) of the photographed image. Moreover, a photographercan designate a subject position on a display screen at any time byoperating an operation member of the imaging apparatus while viewing avideo displayed on the screen of a display unit of the imaging apparatusaccording to a video signal output. In this case, when the photographerperforms an operation of designating a main subject from a plurality ofsubject images displayed on the display screen, the feature amount suchas a hue distribution, a hue size, or the like at the designatedposition is calculated. A region having a similar feature amount to thefeature amount is detected from images obtained sequentially thereafterusing the calculated feature amount, and the detected region can betracked as a main subject region.

Next, a subject tracking control method using the correction lens 108will be described.

The subject detection unit 117 illustrated in FIG. 2 acquires an imagesignal obtained by the imaging element 106 and detects an image position(a subject position) of the subject in the photographed image. Thesubject position information is output to the tracking amountcalculation unit 118. The tracking amount calculation unit 118calculates a control amount used for tracking a subject by the drivingof the correction lens 108 based on the zoom and focus positioninformation 107 and the detected subject position information.Hereinafter, a control amount (tracking control amount) for tracking asubject to position the image of the subject at a predetermined position(a target position (the central position in the present embodiment)) ofthe screen (the range of a photographed image) will be referred to as atracking correction amount. The tracking amount calculation unit 118calculates the tracking correction amount according to the state of thetracking SW 121. The tracking correction amount is input to the adder115 and is added to the blur correction target value which is the outputof the sensitivity adjustment unit 114, such that blur correction andtracking control by the driving of the correction lens 108 are performedsimultaneously.

FIGS. 3A and 3B are diagrams for describing tracking control fortracking the detected main subject according to the present embodiment.FIG. 3A illustrates a photographed image 301 a before subject trackingcontrol starts. FIG. 3B illustrates a photographed image 301 b aftersubject tracking control starts. In the photographed image 301 a of FIG.3A, a black dot illustrated at the center illustrates a screen centralposition 304. An image position of a subject 302 a before the start oftracking control is away from the screen central position 304. A subjectcentral position 303 a illustrates the central position of the subject302 a in the image. When the CPU 105 starts subject tracking control,the distance between the subject central position 303 a and the screencentral position 304 decreases gradually according to the elapse oftime. With the subject tracking control, finally, as illustrated in FIG.3B, the subject central position 303 a becomes substantially identicalto the screen central position 304.

Next, a tracking amount calculation process of the tracking amountcalculation unit 118 will be described with reference to FIG. 4. FIG. 4is a functional block diagram illustrating an example of the trackingamount calculation unit 118. Although the tracking amount calculationunit 118 calculates a tracking correction amount in the respective axesof the vertical and horizontal directions of the screen, only a singleaxis will be described for the sake of simplicity.

A subtractor 403 subtracts the coordinate of the screen central position402 from the correlation of the subject position (the central position)401 based on the subject position information output by the subjectdetection unit 117. In this way, a difference amount (hereinafterreferred to as a center shift amount) indicating the distance betweenthe central position of a subject image and the screen central position402 on the image is calculated. The center shift amount is signed datacalculated when the difference amount at the screen central position 402is 0. The output of the subtractor 403 is input to a count value tablereference unit (hereinafter simply referred to as a reference unit) 404.

The reference unit 404 calculates a count value for tracking based onthe center shift amount (that is, the magnitude of the differenceamount). Specifically, the count value is calculated as follows.

When the center shift amount is equal to or smaller than a predeterminedthreshold A and is equal to or larger than a predetermined threshold“−A,” the count value is set to 0 or a minimum value. When the magnitude(absolute value) of the center shift amount is equal to or smaller thana predetermined threshold, a non-sensitive region in which tracking isnot performed is set to a predetermined range from the screen centralposition.

When the center shift amount is larger than a predetermined threshold Aor is smaller than a predetermined threshold “−A,” the count value isset to increase as the absolute value of the center shift amountincreases. The sign of the count value is calculated according to thesign of the center shift amount.

The output of the reference unit 404 is acquired as a first input of asignal selection unit 406. The signal selection unit 406 acquires a downcount value output by a down count value output unit 405 as a secondinput. Moreover, a signal indicating the state of the tracking SW(switch) 121 is input to the signal selection unit 406 as a controlsignal. When the tracking SW 121 is set to the ON state, the signalselection unit 406 selects the output of the reference unit 404 andoutputs the output to an adder unit 407. Moreover, when the tracking SW121 is set to the OFF state, the signal selection unit 406 selects thedown count value and outputs the down count value to the adder unit 407.The down count value will be described later.

The adder unit 407 acquires the output of the signal selection unit 406and a previous sampling value associated with a tracking amount and addsboth values. The output of the adder unit 407 is input to an upper andlower limit setting unit 408. The upper and lower limit setting unit 408limits the tracking correction amount to be within a predeterminedrange. That is, the tracking correction amount is limited so as not tobe equal to or larger than a predetermined upper limit and not to beequal to or smaller than a predetermined lower limit and the valuethereof is changed. The output of the upper and lower limit setting unit408 is input to a delay unit 409 and a low-pass filter (LPF) 410.

The delay unit 409 outputs a past tracking correction amount (that is, aprevious sampling value) obtained a predetermined sampling periodearlier than the current time point to the adder unit 407 and the downcount value output unit 405 as a calculation result. The down countvalue output unit 405 outputs the down count value. A previous trackingamount sampling value 409 calculated by post-processing is input to thedown count value output unit 405. When the tracking correction amountobtained at a previous sampling time point (the previous sampling value)has a positive sign, the down count value is set to a negative sign.Moreover, when the previous sampling value has a negative sign, the downcount value output unit 405 sets the down count value to a positivesign. In this way, processing is performed so that the absolute value ofthe tracking correction amount decreases. Moreover, when the previoussampling value output by the delay unit 409 is within a range that isnear zero (0±predetermined range), the down count value output unit 405sets the down count value to 0. The down count value is the second inputto the signal selection unit 406.

The LPF 410 removes high-frequency noise included during detection ofthe subject from the output of the upper and lower limit setting unit408 and outputs the processed signal to a correction lens driving amountconversion unit 411. The correction lens driving amount conversion unit411 converts the output of the LPF 410 to a signal for allowing thecorrection lens 108 to perform a subject tracking operation. In thisway, a final tracking correction amount is calculated and the correctionlens 108 is driven based on the tracking correction amount such that atracking correction process is performed so that the central position ofthe subject image is gradually positioned near the center of aphotographed image.

By driving the correction lens 108 based on an addition result of theblur correction amount and the tracking correction amount, added by theadder 115 in the above-described manner, image blur correction controland subject tracking control can be performed simultaneously.

When subject tracking is performed, a photographer can designate asubject within a screen in order to photograph a desired subject. Asubject designation method for tracking will be described by way of aspecific example of FIGS. 5A to 5C. FIG. 5A illustrates an example inwhich a touch panel 502 is provided on a liquid crystal display (LCD)501 provided on a rear surface of the imaging apparatus 101. When aphotographer touches on a display screen, a process of acquiring thecoordinate of a subject 503 at the touched position and setting atracking target subject is executed. Moreover, FIG. 5B illustrates anexample in which an operation SW (switch) 505 for setting a trackingsubject selectable mode (hereinafter referred to as a trackingselectable mode) is provided. When the photographer presses theoperation SW 505, the operation mode enters into a tracking selectablemode. Since an icon 504 for designating a subject is displayed at thecenter of the screen, the photographer can set the subject 503 at thedisplay position of the icon 504 as a tracking target subject byoperating the SW1 of the release switch 104, for example.

In any of the methods illustrated in FIGS. 5A and 5B, when a subject isdesignated, a process of detecting a region having a similar featureamount from images sequentially obtained thereafter using the featureamount such as a hue distribution or a hue size is performed. FIG. 5Cillustrates an example in which a main subject region extracted as aregion having a similar feature amount is presented to a photographer asa tracking region indicated by a frame 506. When a photographerdesignates a subject image within a display screen to track a subject,the following problems may occur before the subject is designated.

For example, when the focal distance of a lens is as large as 1000 mm orlarger, a subject image may move due to the influence of a handshake andit may be difficult to capture a subject within an angle of view toperform framing. In order to alleviate the difficulty of framing due tothe influence of a hand shake, a method of changing characteristics soas to decrease the thrust toward the center of the control range of theoffset subtractor 110 or decreasing the cutoff frequency of the integralfilter unit 111 to increase a blur correction control range may be used.However, when the shake suppression characteristics for hand shakecorrection are enhanced, image blur correction is performed on the handshake amount occurring when the photographer operates the cameraintentionally even when a photographer operates a camera so that thesubject falls within the angle of view. Therefore, due to the influenceof the image blur correction control, it may be difficult to capture asubject within the angle of view again or finely adjust the subjectimage to be positioned near the center of a photographed image. Forexample, before a photographer designates a subject in a state in whichthe operation mode is set to a tracking selectable mode by thephotographer operating the operation SW 505 in FIG. 5B, it is necessaryto allow the photographer to perform framing easily so that the icon 504is moved to the position of the subject 503 in order to set a trackingtarget subject, for example. When the operation mode is set to thetracking selectable mode, the shake suppression characteristics for blurcorrection are enhanced more than that when the tracking selectable modeis not set. In this case, since the photographer is less likely toperform panning greatly in the state in which the tracking selectablemode is set, the panning determination unit 113 increases thedetermination threshold. In this way, the photographer can easilyperform panning so that the icon 504 is positioned at the position ofthe subject 503.

After the subject is designated, correction for capturing a subjectimage at a specific position (for example, the central position) withina photographed image by tracking control is performed simultaneouslywith image blur correction. In this case, shake suppressioncharacteristics associated with image blur correction are enhanced tosuppress the influence of a hand shake as much as possible. In thiscase, the shake suppression characteristics are enhanced further thanthat before the subject is designated. When tracking of the designatedsubject starts, since the photographer is still less likely to performpanning greatly, the panning determination unit 113 increases thedetermination threshold to be larger than that before the subject isdesignated.

In the following description, a default state in which the trackingselectable mode is not set will be referred to as a first state, and astate in which the tracking selectable mode is set, the subject is notselected, and the tracking SW 121 is in the OFF state will be referredto as a second state. A state in which the tracking selectable mode isset, the subject is selected, and the tracking SW 121 is in the ON statewill be referred to as a third state (a tracking mode setting state).The image blur correction effect in the second state is stronger thanthe image blur correction effect in the first state. Also, the imageblur correction effect in the third state is stronger than the imageblur correction effect in the second state. The cutoff frequency of theintegral filter unit 111 in the first state will be denoted by fc1 and apanning determination threshold will be denoted by Ah1. The cutofffrequency of the integral filter unit 111 in the second state will bedenoted by fc2 and a panning determination threshold will be denoted byAh2. The cutoff frequency of the integral filter unit 111 in the thirdstate will be denoted by fc3 and a panning determination threshold willbe denoted by Ah3. The cutoff frequencies and the panning determinationthresholds satisfy the following relations represented by Expressions(2) and (3).

fc1>fc2>fc3   (2)

Ah1<Ah2<Ah3   (3)

The degree of the image blur correction effect changes from the firststate in which the tracking selectable mode is not set to the secondstate in which a tracking operation is not performed, and to the thirdstate in which the tracking operation is performed. Therefore, it ispossible to perform optimal image blur correction control whendesignating a subject and tracking the subject and to performphotographing while smoothly tracking the subject.

Next, a process of changing the degree of the image blur correctioneffect according to a subject detection position will be described. Whena trackable range of the subject tracking control is sufficient,tracking control is performed so that the subject image is moved nearthe center of a screen. However, since the trackable range is limited,further tracking control cannot be performed when a control positionreaches the end (limit position) of the trackable range. In that case,the photographer performs framing of the camera so that the subjectimage is moved near the center of the screen. In this case, when imageblur correction is performed on a hand shake amount occurring when thephotographer operates the camera intentionally, an adverse effect mayoccur in correction. That is, it may be difficult for the photographerto capture a subject within the angle of view again or finely adjust theposition of a subject image to be positioned at the center of aphotographed image. Therefore, in the present embodiment, a process ofchanging the cutoff frequency (fc4) of the integral filter unit 111according to a subject detection position detected by the subjectdetection unit 117 is performed. A process example will be describedwith reference to FIG. 6.

FIG. 6 illustrates a graph illustrating setting the cutoff frequency fc4(vertical axis) according to a subject detection position (horizontalaxis). The coordinate of a subject detection position is detected in thetwo axial directions of the vertical and horizontal directions of thescreen. With regard to lens control, the cutoff frequency fc4 of theintegral filter unit 111 corresponding to the pitch control in thevertical direction and the yaw control in the horizontal direction isset. Moreover, it is assumed that the subject detection positionindicates the position of a subject in a coordinate in which the centralposition (the target position) of an angle of view is set to zero.

FIG. 6 illustrates a table having such characteristics that the fartherthe subject position located away from the central position of the angleof view, the larger the cutoff frequency fc4. P1 and P2 are thresholdsof a positive-value range and −P1 and −P2 are thresholds of anegative-value range. When the subject detection position is within therange from −P1 to +P1, the cutoff frequency fc4 is set to a defaultvalue D1. When the subject detection position is equal to or larger thanP2 or equal to or smaller than −P2, the cutoff frequency fc4 is set to afrequency D2. Here, D2>D1. When the subject detection position is in asegment between P1 and P2 or a segment between −P1 and −P2, the cutofffrequency fc4 is set to a value calculated by linearly interpolatingbetween D1 and D2 using a linear equation.

The blur prevention characteristics changing unit 112 compares thecalculated cutoff frequency fc4 with the cutoff frequency of theintegral filter unit 111 which has been determined according to thetracking selectable mode and the state of the tracking SW 121. As aresult of comparison, the higher cutoff frequency is set as a finalcutoff frequency of the integral filter unit 111.

By changing the cutoff frequency of the integral filter unit 111according to the subject position in this manner, the image blurcorrection effect when the subject image is positioned near the centerof the screen is enhanced, and it is possible to prevent the subjectimage from deviating from the center of the screen due to the influenceof a hand shake. Moreover, when the subject image is away from thecenter of the screen, a state in which it is not possible to move thesubject image to the center of the screen by tracking control (a statein which the trackable range is near a control end, for example) iscreated. The control end is a position at which the realization ofcorrection is limited. In this case, the image blur correction effect isslightly weakened so that a photographer can move the subject image tobe positioned near the center of the screen by framing. That is, it ispossible to prevent image blur correction from being performed byframing of the photographer and to facilitate framing.

A process example of the blur correction control and the subjecttracking control will be described with reference to the flowchartillustrated in FIG. 7. The following process starts when a main powersupply of the imaging apparatus 101 is turned on and is executed at afixed sampling cycle.

First, in S710, the CPU 105 determines whether a blur prevention SW(switch) is in the ON state. The blur prevention SW is an operationswitch indicating whether the user performs image blur correctioncontrol. When the blur prevention SW is in the ON state, the flowproceeds to S702. When the blur prevention SW is in the OFF state, theflow proceeds to S703. In S702, the CPU 105 acquires the output of theangular velocity meter 103 and the flow proceeds to S704. In S703, theoutput of the angular velocity meter 103 is not acquired, and the flowproceeds to S704 after the angular velocity is set to zero. When theangular velocity is set to zero, a subsequent blur correctioncalculation result (that is, a blur correction amount) becomes zero.

In S704, the CPU 105 determines whether the tracking selectable mode isset to ON. For example, it is determined whether the tracking selectablemode is set to ON by the operation SW 505 in FIG. 5B. When the trackingselectable mode is set to ON, the flow proceeds to S705. When thetracking selectable mode is set to OFF, the flow proceeds to S708. InS705, the CPU 105 determines whether the tracking SW 121 is in the ONstate. Whether the tracking SW 121 is in the ON or OFF state can bedetermined based on whether a subject is selected by the methoddescribed in FIGS. 5A to 5C, for example. When the tracking SW 121 is inthe ON state, the flow proceeds to S706. When the tracking SW 121 is inthe OFF state, the flow proceeds to S707.

In S706, the cutoff frequency (denoted by fa) of the integral filterunit 111 is set to fc3. Moreover, The panning determination unit 113sets the panning determination threshold (denoted by Tha) to Ah3 and theflow proceeds to S709. In S709, the subject detection unit 117 detects asubject position in a photographed image. In this example, it is assumedthat at the time point at which the tracking SW 121 is set to ON inS705, a subject image on the LCD screen on which a captured image isdisplayed in a live view is designated by a user's operation. That is,the imaging apparatus 101 recognizes the subject and acquires thecoordinate of the detection position of the subject by tracking anddetecting the subject.

Subsequently, in S710, the tracking amount calculation unit 118calculates a tracking correction amount from the detected subjectposition and the flow proceeds to S711. In S711, the CPU 105 calculatesthe cutoff frequency fc4 of the integral filter unit 111 based on thedetected subject position. In S712, the CPU 105 determines whether thecurrent cutoff frequency fa is smaller than fc4. When fa<fc4, the flowproceeds to S713. When fa≥fc4, the flow proceeds to S715. In S713, theCPU 105 substitutes the value fc4 into fa, and the flow proceeds toS715.

In S708, the CPU 105 sets the cutoff frequency fa of the integral filterunit 111 to fc1. The panning determination unit 113 sets the panningdetermination threshold Tha to Ah1 and the flow proceeds to S714. InS707, the CPU 105 sets the cutoff frequency fa of the integral filterunit 111 to fc2. The panning determination unit 113 sets the panningdetermination threshold Tha to Ah2 and the flow proceeds to S714. Afterthe tracking correction amount is set to zero in S714, the flow proceedsto S715.

In S715, the blur correction angle calculation unit 109 calculates ablur correction amount. In this case, the angular velocity acquired inS702 or S703, and the cutoff frequency fa of the integral filter unit111 and the panning determination threshold Tha of the panningdetermination unit 113 set in S705 to S703 are used. After the blurcorrection amount is calculated, the flow proceeds to S716. In S716, theadder 115 adds the blur correction amount and the tracking correctionamount to calculate a lens driving amount. Subsequently, in S717, theCPU 105 controls the driving of the correction lens 108 based on thelens driving amount with the aid of the driving control unit 116. Inthis way, image blur correction and subject tracking control areperformed. When the image blur correction process and the subjecttracking control process end, a standby is performed until a subsequentsampling time point arrives.

An image position control device of the present embodiment calculates asubject tracking amount based on a subject position in a photographedimage. The subject tracking amount is a control amount for moving asubject image to be positioned at a specific position (for example, thecentral position or a position designated by a user) within a screen.Moreover, the image position control device performs calculation basedon an angular velocity output as a blur detection signal to calculate ablur correction amount (image blur correction amount). The blurcorrection amount and the subject tracking amount are added and imageblur correction and subject tracking control are performed bycontrolling the driving of a correction lens based on the added value.In this case, the characteristics of calculating the blur correctionamount are changed according to a control state of the subject tracking.That is, a process of changing the degree of an image blur correctioneffect based on determination result information indicating whether thetracking selectable mode is set and the ON/OFF setting information ofthe tracking switch and a process of changing the degree of an imageblur correction effect according to the subject position are performed.Therefore, it is possible to realize image blur correction control whichallows a photographer to easily perform panning, and to improve theperformance of the image blur correction control and the automaticsubject tracking control.

In the present embodiment, an application example to so-called opticalblur prevention control in which a correction lens is used as an imageblur correction and automatic subject tracking control unit and is movedwithin a plane vertical to an optical axis has been described. However,the present invention can be applied to an image position control devicehaving the following configurations.

(1) A configuration for moving an imaging element within a planevertical to an optical axis.

(2) A configuration in which electronic control is performed by imageprocessing for changing a slicing position of photographic frames outputby an imaging element.

(3) A configuration having a mechanism for rotating a lens barrelincluding an imaging element and a photographing lens group.

(4) A configuration combined with a driving mechanism (for example, arotary pan head for performing panning or tilting of an imagingapparatus) provided separately from the imaging apparatus.

(5) A combination of a plurality of configurations described in (1) to(4).

This is also true for a third embodiment to be described later.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe present embodiment, image blur correction and automatic subjecttracking when a mechanism for rotating a lens barrel including aphotographing lens group and the imaging element 106 as well as thecorrection lens 108 is included will be described. In the presentembodiment, the same constituent elements as those of the firstembodiment will be denoted by the previously used reference numerals,the detailed description thereof will not be provided and the differencewill be mainly described. The same is true for the embodiments to bedescribed later.

FIGS. 8A and 8B are diagrams schematically illustrating an imagingapparatus 101 according to the present embodiment. In FIGS. 8A and 8B,among the X, Y, and Z-axes of a three-dimensional orthogonal coordinatesystem, a direction around the X-axis is defined as a pitch directionand a direction around the Y-axis is defined as a yaw direction. TheZ-axis direction is the optical axis direction of the imaging apparatus101. The imaging apparatus 101 illustrated in FIG. 8A includes anoperating member such as the release switch 104 and a display unit thatdisplays a photographed image. The display unit which uses an LCD or thelike has a monitor function of displaying a photographed image in a liveview on a realtime basis. A lens barrel 801 includes a photographinglens group and the imaging element 106 and is attached in a state ofbeing drivable by the imaging apparatus 101. That is, a mechanism forrotating (tilting) the lens barrel 801 in relation to the imagingapparatus 101 is provided.

FIG. 8B illustrates a structure associated with rotation of the lensbarrel 801. A driving mechanism 802 p includes a motor that rotates thelens barrel 801 in the pitch direction and a driving control unitthereof. A driving mechanism 802 y includes a motor that rotates thelens barrel 801 in the yaw direction and a driving control unit thereof.The attitude of the lens barrel 801 can be controlled in the pitchdirection and the yaw direction independently by the driving mechanisms802 p and 803 y.

FIG. 9 is a diagram illustrating a configuration of a main part of theimaging apparatus 101. A difference between this configuration and theconfiguration of the first embodiment described in FIG. 2 will bedescribed below.

(1) The rotation driving mechanism 802 for rotating the lens barrel 801using a motor is provided.

(2) The adder 115 that adds the blur correction amount and the trackingcorrection amount is removed, and the blur correction amount output bythe sensitivity adjustment unit 114 is input to the driving control unit116.

(3) A driving control unit 901 for driving the rotation drivingmechanism 802 is added and the tracking correction amount output by thetracking amount calculation unit 118 is input to the driving controlunit 901.

(4) The driving control unit 901 performs automatic subject trackingcontrol by driving the rotation driving mechanism 802 based on thetracking correction amount calculated by the tracking amount calculationunit 118.

The image position control device of the present embodiment performsimage blur correction using the correction lens 108 and performsautomatic subject tracking control using the rotation driving mechanism802. In this case, the same advantages as those of the first embodimentare obtained, and it is possible to perform image blur correctioncontrol which allows the photographer to easily perform framing.

In the present embodiment, the angular velocity meter 103 is attached tothe lens barrel 801 or is attached to a portion of the imaging apparatus101 other than the lens barrel 801. When the angular velocity meter 103is attached to the lens barrel 801, a relative angular velocity of thelens barrel 801 in relation to a fixing portion of the imaging apparatus101 is subtracted from the output of the angular velocity meter 103. Therelative angular velocity between the imaging apparatus 101 and the lensbarrel 801 is a rotating velocity of the lens barrel 801 rotated by therotation driving mechanism 802 and is detected based on a drivinginstruction signal of a motor or by a rotation detection sensor or thelike. By subtracting the relative angular velocity from the output ofthe angular velocity meter 103, it is possible to calculate a shakeamount of the imaging apparatus 101.

In the present embodiment, a correction lens and a driving mechanismthereof are used as an image blur correction unit, and the lens barrel801 including an imaging element and a photographing lens group and arotation driving mechanism thereof are used as an automatic subjecttracking unit. The present invention is not limited thereto, and thefollowing configuration can be applied.

(1) A configuration including a mechanism for moving an imaging elementwithin a plane vertical to an optical axis and a mechanism for driving acorrection lens.

(2) A configuration including a processing unit that changes an imageslicing position of respective photographic frames output by an imagingelement and a mechanism for driving a correction lens.

(3) A configuration including a processing unit that changes an imageslicing position of respective photographic frames output by an imagingelement and a mechanism for moving the imaging element within a planevertical to an optical axis.

(4) A configuration including a mechanism for moving an imaging elementwithin a plane vertical to an optical axis and a mechanism for rotatinga lens barrel including a photographing lens group.

(5) A configuration including a processing unit that changes an imageslicing position of respective photographic frames output by an imagingelement and a mechanism for rotating a lens barrel including aphotographing lens group.

(6) A combination of a plurality of configurations described in (1) to(5).

Third Embodiment

Next, a third embodiment of the present invention will be described. Animage blur correction and automatic subject tracking device of thepresent embodiment changes the characteristics of image blur correctionbased on subject detection information used for automatic subjecttracking. The subject detection information is information indicating asubject detection state such as whether a subject is being detected orthe subject has been lost and is information on reliability of subjectdetection. Hereinafter, image blur correction and automatic subjecttracking control which takes the subject detection information intoconsideration will be described.

FIG. 10 is a diagram illustrating a configuration of a main part of animaging apparatus according to the present embodiment. A differencebetween this configuration and the configuration of the first embodimentdescribed in FIG. 2 will be described below.

(1) The subject detection unit 117 includes a subject detection statedetermination unit 1001 and a subject reliability determination unit1002 in addition to a subject position detection unit 1000, anddetection information and determination information are input to atracking gain unit 1003, the blur prevention characteristics changingunit 112, and the tracking amount calculation unit 118.

(2) The tracking gain unit 1003 is added, and a gain is multiplied withthe tracking correction amount which is the output of the trackingamount calculation unit 118 and the multiplied tracking correctionamount is input to the adder 115.

The items of determination information obtained by the subject detectionstate determination unit (hereinafter referred to as a detection statedetermination unit) 1001 and the subject reliability determination unit(hereinafter referred to as a reliability determination unit) 1002 areinput to the tracking gain unit 1003, and the tracking gain unit 1003sets a gain according to the respective determination results. The gainis set to 0 or more and 1 or smaller. The tracking gain unit 1003 andthe blur prevention characteristics changing unit 112 will be describedin detail below.

The subject position detection unit 1000 corresponds to the subjectdetection unit 117 of the first embodiment and detects a subjectposition using face detection or pattern matching, for example. Thedetection state determination unit 1001 determines whether the subjectis in a first detect state or a second detect state. The first detectstate is a state in which a subject has been detected. The second detectstate is a state (subject lost state) in which a subject is not detectedbecause the subject is lost. The state can be determined based onwhether the subject position detection unit 1000 has succeeded indetection, for example.

Since the tracking control can be performed in the first detect state,the tracking gain unit 1003 sets the gain to 1. That is, the trackingcorrection amount calculated by the tracking amount calculation unit 118is input to the adder 115 as it is. Moreover, since subject trackingcontrol is performed, the image blur correction effect is enhanced. Asdescribed in the first embodiment, the characteristics of the offsetsubtractor 110 and the integral filter unit 111 are set according to thestate of the tracking SW 121 and the determination result of the panningdetermination unit 113, and the blur correction amount is calculated.

On the other hand, when the detection state determination unit 1001 hasdetermined that the subject is in the second detect state, it ispreferable to perform control to stop subject tracking. That is, when itis determined that the subject is in the second detect state, it ispreferable to set the gain to 0. Moreover, the detection statedetermination unit 1001 may determine a subject detection state fromthree or more states rather than the two states of the first and seconddetect states. For example, the first detect state is defined as a statein which it is highly likely that the subject has been detected, and thesecond detect state is defined as a state in which it is highly likelythat the subject has not been detected. Moreover, the third detect stateis a state in which it is less likely that the subject has been detectedthan the first detect state and it is more likely that the subject hasbeen detected than the second detect state. For example, during apredetermined period of time, the detection state determination unit1001 detects a state in which is lost but continues to search for thesubject (third detect state) and if it detects a subject again duringthe subject lost state, shifts to the first state, which is the subjectdetection again state. When the subject lost state continues for apredetermined period of time, the detection state determination unit1001 shifts the second state, which is a subject non-detection state,after stopping the search for the subject. For example, the third statemay be set so that the possibility of the object being detected by theelapsed time from when the subject was lost. In this case, the trackinggain unit 1003 sets the gain to 1 when the subject is in the firstdetect state, sets the gain to 0 when the subject is in the seconddetect state, and sets the gain to a value larger than 0 and smallerthan 1 when the subject is in the third detect state.

The detection state determination unit 1002 may determine a detectionstate from a plurality of states set according to the possibility thatthe subject is detected between the first and second detect states. Inthis case, the tracking gain unit 1003 sets the gain so as to graduallydecrease as the possibility that the subject is detected decreases andfinally reach zero. Since tracking control is not performed in thesecond detect state, it is set such that the subject image can be movedto be positioned near the center of the screen by the framing operationof the photographer. That is, the blur prevention characteristicschanging unit 112 slightly weakens the image blur correction effect,prevents image blur correction from being performed on a framingoperation, and changes the blur prevention characteristics so that thephotographer can easily perform framing. In this case, the blurprevention characteristics facilitate framing while maintaining theimage blur correction effect to a certain strength.

The cutoff frequency of the integral filter unit 111 is set to fc2 andthe panning determination threshold is set to Ah2. This setting is thesame as that when the subject is not designated and the tracking SW isin the OFF state illustrated in S707 of FIG. 7. The blur preventioncharacteristics changing unit 112 changes the blur preventioncharacteristics according to the determination result on the subjectdetection state obtained by the detection state determination unit 1001,the state of the tracking SW 121, and the determination result of thepanning determination unit 113. That is, the blur preventioncharacteristics which best decrease the image blur correction effect aredetermined among the blur prevention characteristics corresponding tothe determination result on the subject detection state and the blurprevention characteristics corresponding to the state of the tracking SW121 and the determination result of the panning determination unit 113.The offset subtractor 110 and the integral filter unit 111 are setaccording to the determined blur prevention characteristics.

The reliability determination unit 1002 determines whether the detectedsubject is reliable. The reliability of the subject can be determinedbased on the size of a subject image in a photographed image, forexample. Alternatively, the reliability of a main subject which is adetermination target can be calculated based on the degree of matchingbetween the pattern of a subject image stored when designating a subjectand the detected subject image. Moreover, since it is highly likely thata false main subject has been detected even when there are a pluralityof subjects having the same pattern as the detected subject, thereliability is calculated to be low. The reliability determination unit1002 compares the calculated reliability with a threshold to determinewhether the detected subject is reliable. When the reliability of thesubject is high (equal to or higher than the threshold), subjecttracking control can be performed. In this case, the tracking gain unit1003 sets the gain to 1. The tracking correction amount calculated bythe tracking amount calculation unit 118 is input to the adder 115 as itis. In order to increase the image blur correction effect during thetracking control, the characteristics of the offset subtractor 110 andthe integral filter unit 111 are changed and the blur correction amountis calculated.

On the other hand, when the reliability determination unit 1002determines that the reliability of the subject is smaller than thethreshold and the reliability is low, control is performed to stoptracking since it is likely that the tracking control may be undesiredby the photographer. The reliability determination unit 1002 preferablyevaluates the reliability in multiple steps using a plurality ofthresholds rather than evaluating whether the reliability is reliableusing the same threshold in two steps. In this case, when the calculatedreliability is equal to or higher than the largest threshold (thereliability is evaluated to be the highest), the tracking gain unit 1003sets the gain to 1 and gradually decreases the gain according to thereliability. Moreover, when the reliability is smaller than the smallestthreshold, the gain is finally set to zero. In this case, the blurprevention characteristics are changed to slightly weaken the image blurcorrection effect. That is, the cutoff frequency of the integral filterunit 111 is set to fc2 and the panning determination threshold is set toAh2. In this case, the blur prevention characteristics facilitateframing while maintaining the image blur correction effect to a certainstrength. The blur prevention characteristics changing unit 112 changesthe blur prevention characteristics according to the determinationresult of the reliability determination unit 1002, the determinationresult on the subject detection state obtained by the detection statedetermination unit 1001, the state of the tracking SW 121, and thedetermination result of the panning determination unit 113. That is, theblur prevention characteristics corresponding to the subject reliabilitywhich is the determination result of the reliability determination unit1002, the blur prevention characteristics corresponding to the subjectdetection state, and the blur prevention characteristics correspondingto the state of the tracking SW 121 and the determination result of thepanning determination unit 113 are compared. Among these blur preventioncharacteristics, the blur prevention characteristics which best decreasethe image blur correction effect are determined. The offset subtractor110 and the integral filter unit 111 are set according to the determinedblur prevention characteristics.

The tracking correction amount output from the tracking gain unit 1003is input to the adder 115 and is added to the target value of the blurcorrection amount which is the output of the sensitivity adjustment unit114. In this way, tracking correction and image blur correction can beperformed simultaneously by driving the correction lens 108.

In the present embodiment, the characteristics of image blur correctionare changed according to the subject detection state or the subjectreliability or both. Therefore, since it is possible to realize imageblur correction control which allows the photographer to easily performframing, it is possible to improve the performance of the image blurcorrection and automatic subject tracking control.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Animage blur correction and automatic subject tracking device according tothe present embodiment determines whether tracking control can beperformed based on subject detection information used for automaticsubject tracking and displays a determination result on the screen of adisplay unit. In this way, it is possible to notify whether the imagingapparatus can track a subject to the photographer and to realize imageblur correction and subject tracking control which allows thephotographer to easily perform framing.

In a system that tracks a subject so that the position of the subject ismaintained at a specific position of a photographing screen, sinceduring a tracking operation, the camera tracks a subject so that thesubject is automatically positioned near the center of the angle of viewby the tracking control, the photographer does not need to perform aframing operation to track the subject. However, such subject trackingis performed by moving a portion of an optical system, there is a limitin the trackable range. Since further tracking cannot be performed whenthe optical system reaches the end of the trackable range, thephotographer needs to shake the camera to track the subject. However,since it is difficult for the photographer to immediately recognize thatthe optical system has reached the end of the trackable range, thephotographer may lose the subject from the photographing screen due to alate framing operation. This may occur even when tracking is performedby changing the slicing range of a frame.

In the present embodiment, it is determined whether the camera canperform tracking control and the determination result is displayed onthe screen of a display unit. In this way, it is possible to improve theoperability of the photographer's framing operation.

FIG. 11 is a diagram illustrating a configuration of a main part of animaging apparatus according to the present embodiment. Thisconfiguration is different from the configuration described in FIG. 10in the third embodiment in that this configuration includes a trackingstate determination unit 120 and a tracking icon control unit 123. Adetermination result obtained by the tracking state determination unit120 is input to the tracking icon control unit 123, and an icon to bedisplayed on the display unit 122 is controlled by the tracking iconcontrol unit 123.

When subject tracking control is performed and the trackable range issufficient, tracking control is realized so that the position of thesubject is maintained to be near the center of the screen. However,since the trackable range is limited, further tracking control cannot beperformed when the optical system reaches the end of the trackablerange. In this case, the photographer needs to frame the camera again tomove the subject to be positioned near the center. However, it isdifficult for the photographer to immediately determine whether furthertracking can be performed just by viewing a live-view image. Thus, thephotographer may not perform framing again immediately after thecorrection lens 108 reaches the end of the trackable range, and thesubject may be lost from the screen. Therefore, the imaging apparatus101 of the present embodiment determines whether tracking control can beperformed. Moreover, the tracking icon to be displayed on the screen ischanged according to the determination result and the photographer isnotified whether tracking can be performed. In this way, when trackingcannot be performed, a warning notification is issued to urge thephotographer to perform an operation of tracking the subject by aframing operation.

FIGS. 12A to 12H illustrate a camera rear view illustrating a subjecttracking operation and the tracking state and timing charts illustratingthe control state. When subject tracking is performed, in order for thephotographer to photograph a desired subject, it is preferable for thephotographer to designate a subject in the screen and perform tracking.As a method of designating a subject to be tracked, for example, asillustrated in FIG. 12A, a method of acquiring a coordinate position ofa touched subject 503 with the aid of the touch panel 502 provided underthe LCD 501 on the rear surface of the camera 101 and setting a trackingtarget subject may be used.

When a subject is designated by the method illustrated in FIG. 12A, amain subject region is tracked by detecting regions having a similarfeature amount from images sequentially obtained thereafter by alive-view operation using the feature amount such as a hue distributionor a hue size. Moreover, the main subject region is depicted as atracking region by the frame 506 to notify the photographer of thetracking region. After the subject is designated, tracking control isperformed so that the detected subject position is moved toward thecenter of the screen. Moreover, a tracking icon 505 is displayed on thescreen so that the photographer understands that the tracking control isperformed. When subject tracking is not performed (the subject is notdesignated), the tracking icon 505 is not displayed on the screen.

FIG. 12B is a diagram illustrating a state after the subject isdesignated. FIG. 12F illustrates the output of the adder 115 obtained byadding the blur correction amount and the tracking correction amountwhen the state of the screen displayed by the imaging apparatus 101changes from FIG. 12A to FIG. 12B, FIG. 12C, FIG. 12D, and FIG. 12Eaccording to the elapse of time. FIG. 12G illustrates a center shiftamount which is the output of the subtractor 403 and FIG. 12Hillustrates a display state of the tracking icon 505. When the subjectposition detection unit outputs the coordinate of the detected subjectusing the target position as a central coordinate, the output of thesubject position detection unit can be regarded as a center shiftamount. The tracking icon 505 is not displayed on the screen untiltiming T1 at which the subject is designated, and the tracking icon 505is displayed on the screen when the subject is designated at timing T1as illustrated in FIG. 12B. Subsequently, even when the subject 503 isaway from the center of the image when the designated subject 503 moves(FIG. 12C) , the tracking control is performed based on the trackingcorrection amount calculated by the tracking amount calculation unit 118such that the subject 503 returns to the center of the image (FIG. 12D).In the drawings, T2 indicates the timing at which the subject moves awayfrom the center of the image and T3 indicates the timing at which thesubject returns to the center of the image by the tracking control.

Subsequently, although tracking control is performed while furtherincreasing the tracking correction amount when the subject movesfurther, the tracking correction amount is limited so as not to exceed acorrectable amount Th2 when the correction amount of the correction lens108 reaches a trackable threshold Th1 as illustrated in FIG. 12F. Inthis case, the correction amount is set to be between Th1 and Th2, andthe tracking icon 505 is displayed in gray at T5 at which a state inwhich the correction amount is equal to or larger than the threshold Th1has continued for a predetermined period TL or more in order to notifythe photographer of the fact that further tracking control cannot beperformed. That is, the CPU 105 functioning as the tracking statedetermination unit and the tracking icon control unit determines thatthe tracking control cannot be performed when the state in which thecorrection amount is the threshold Th1 or more has continued for thepredetermined period TL or more, displays the tracking icon 60 in gray,and issues a warning instruction to send a warning to the photographer.The correctable amount Th2 is an amount required for the correction lensto reach the end of the movable range and the trackable threshold Th1can be appropriately set to a value smaller than Th2. For example, thetrackable threshold Th1 may be set to a value obtained by multiplyingthe correctable amount Th2 by a coefficient larger than 0 and smallerthan 1 and may be set to a value obtained by subtracting a fixed amountfrom the correctable amount Th2.

Since further tracking control cannot be performed when the correctionlens 108 reaches the end of the movable range, the display of thetracking icon 505 is changed in the above-described manner to urge thephotographer to frame the camera to move the subject toward the centerof the image.

Next, another example of a method of sending a notification to thephotographer by changing the tracking icon 505 when the correction lens108 exceeds the trackable range and the tracking control cannot beperformed will be described with reference to FIGS. 13A to 13H. A methodof determining whether the tracking operation can be performed used inthe example of FIGS. 13A to 13H is different from the example of FIGS.12A to 12H. When it is not possible to move the position of the subjectto the vicinity of the center of the image after the start of trackingcontrol and the subject is positioned at the position away from thecenter for a long period, it can be determined that the trackingcorrection has reached the end of the movable range and further trackingcontrol cannot be performed. In this case, a warning icon is displayedto notify the fact that tracking control cannot be performed.

Whether the tracking operation can be performed or not is determined inthe following manner. That is, when a period in which a center shiftamount which is the distance on an image between the subject positionand the center of the image exceeds a predetermined threshold Th3 isequal to or longer than a predetermined period T7, it is determined thattracking cannot be performed. In other words, when a state in which theposition of a subject image is separated by a predetermined distance ormore from a target position has continued for a predetermined period orlonger, it is determined that the subject cannot be tracked. Moreover,when the center shift amount is equal to or smaller than the thresholdTh3, it is determined that tracking can be performed. This determinationis performed by the tracking state determination unit 120. The trackingicon displayed on the screen is changed according to the determinationresult such that the photographer is notified of the information onwhether tracking can be performed. When tracking cannot be performed,the photographer is urged to perform a framing operation to track thesubject.

Similarly to the method described in FIG. 12A, when a subject isdesignated by a touch operation in FIG. 13A, the tracking control startsand the tracking icon 505 is displayed on the screen.

FIGS. 12B to 12E are diagrams illustrating the state after the subjectis designated. FIG. 12F illustrates the output of the adder 115 obtainedby adding the blur correction amount and the tracking correction amountwhen the state of the screen displayed by the imaging apparatus 101changes from FIG. 12A to FIG. 12B, FIG. 12C, FIG. 12D, and FIG. 12Eaccording to the elapse of time. FIG. 12G illustrates a center shiftamount and FIG. 12H illustrates a display state of the tracking icon505. The tracking icon 505 is not displayed on the screen until timingT8 at which the subject is designated, and the tracking icon 505 isdisplayed on the screen when the subject is designated at timing T8.Subsequently, even when the subject 503 is away from the center of theimage when the designated subject 503 moves (FIG. 12C) , the trackingcontrol is performed based on the tracking correction amount calculatedby the tracking amount calculation unit 118 such that the subject 503returns to the center of the image (FIG. 12D). In the drawings, T9indicates the timing at which the subject moves away from the center ofthe image and T10 indicates the timing at which the subject returns tothe center of the image by the tracking control. Although the subjectposition exceeds the threshold Th3 in the period between T9 and T10,since the period in which the position exceeds Th3 does not exceed apredetermined period T7, the icon 505 is displayed in a normal color.

Subsequently, although tracking control is performed while furtherincreasing the tracking correction amount when the subject movesfurther, the tracking icon 505 is displayed in gray in order to notifythe photographer of the fact that further tracking control cannot beperformed when a state in which the center shift amount is equal to orlarger than the threshold Th3 as illustrated in FIG. 12G continues forthe predetermined period T7 or longer.

In this manner, the CPU 105 functioning as the tracking statedetermination unit 120 determines that tracking has reached the end ofthe movable range and further tracking control cannot be performed whenthe subject position is away from the center of the image for a longperiod. Moreover, the CPU 105 also functions as the tracking iconcontrol unit 123 and changes the display of the tracking icon 505 tourge the photographer to frame the camera to move the subject toward thecenter of the image.

In FIGS. 12A to 12H and FIGS. 13A to 13H, an example in which an icon isdisplayed when it is difficult to perform subject tracking control sincethe subject in the image moves a large distance and a correction amountfor moving the subject position to the target position in the image islarge has been described. However, tracking control may be difficulteven when the correction amount is small. An example of a method ofnotifying the photographer by changing the display of the tracking icon505 when tracking control cannot be performed due to a subject detectionstate will be described with reference to FIGS. 14A to 14H. When thereliability of the detected subject is low, it can be determined thattracking control cannot be performed. In this case, an icon is displayedto indicate that tracking control cannot be performed.

The reliability is determined by the subject reliability determinationunit 1002 and it is determined whether the detected subject is reliable.Since the subject reliability determination by the subject reliabilitydetermination unit 1002 is the same as that of the third embodiment, thedetailed description thereof will not be provided.

Similarly to the method described in FIG. 12A, when a subject isdesignated by a touch operation in FIG. 14A, tracking control starts andthe tracking icon 505 is displayed on the screen.

FIG. 14B is a diagram illustrating a state after the subject isdesignated. FIG. 14E illustrates the output of the adder 115 obtained byadding the blur correction amount and the tracking correction amountwhen the state of the screen displayed by the imaging apparatus 101changes from FIG. 14A to FIG. 14B, FIG. 14C, and FIG. 14D according tothe elapse of time. Moreover, FIG. 14F illustrates a center shiftamount, FIG. 14G illustrates the reliability of a subject, and FIG. 14Hillustrates a display state of the tracking icon 505. The tracking icon505 is not displayed on the screen until timing T11 at which the subjectis designated, and the tracking icon 505 is displayed on the screen whenthe subject is designated at timing T11. Subsequently, even when thedesignated subject 503 is away from the center of the image, trackingcontrol is performed based on the tracking correction amount calculatedby the tracking amount calculation unit 118 such that the subject 503returns to the center of the image as illustrated in FIG. 14B. In thedrawings, T11 indicates the timing at which a subject is designated andthe subject position is detected and T12 indicates the timing at whichthe subject returns to the center of the image by the tracking control.

Subsequently, as illustrated in FIG. 14C, although tracking control isperformed while further increasing the tracking correction amount whenthe subject moves further, the reliability of the subject is set to below when a plurality of similar subjects appear in the screen asillustrated in FIG. 14D. When the subject reliability decreases, thegain of the tracking gain unit 1003 is set to be small and trackingcontrol is not performed. Therefore, as illustrated in FIG. 14G, whenthe subject reliability is equal to or smaller than a predeterminedthreshold Th4, the tracking icon 505 is displayed in gray to notify thephotographer of the fact that further tracking control cannot beperformed.

In this manner, the CPU 105 functioning as the tracking statedetermination unit 120 determines that tracking control cannot beperformed when the subject reliability is low. Moreover, the CPU 105also functions as the tracking icon control unit 123 and changes thedisplay of the tracking icon 505 to urge the photographer to frame thecamera to move the subject toward the center of the image.

Next, another example of a method of notifying the photographer bychanging the display of the tracking icon 505 when tracking controlcannot be performed due to a subject detection state will be describedwith reference to FIGS. 15A to 15H. Although tracking control isperformed when a subject is detected after the subject is designated,the state transitions to a subject lost state when the photographerloses the subject. Although the state transitions to the subjectdetection state when the subject can be detected again in the subjectlost state, it is determined that tracking control cannot be performedif the subject lost state continues for a predetermined period. In thiscase, an icon is displayed to notify the fact that the tracking controlcannot be performed. Moreover, when the subject lost state continues fora still longer period, it is determined that there is no possibility todetect the subject again, subject detection is stopped, and the trackingicon 505 is not displayed. The subject detection state is determined bythe subject detection state determination unit 1001 based on whether thesubject is detected, the subject is lost, or the subject is notdetected.

Similarly to the method described in FIG. 12A, when a subject isdesignated by a touch operation in FIG. 15A, tracking control starts andthe tracking icon 505 is displayed on the screen.

FIG. 15B is a diagram illustrating a state after the subject isdesignated. FIG. 15E illustrates the output of the adder 115 obtained byadding the blur correction amount and the tracking correction amountwhen the state of the screen displayed by the imaging apparatus 101changes from FIG. 15A to FIG. 15B, FIG. 15C, and FIG. 15D according tothe elapse of time. Moreover, FIG. 15F illustrates a center shiftamount, FIG. 15G illustrates the subject detection state, and FIG. 15Hillustrates a display state of the tracking icon 505.

The tracking icon 505 is not displayed on the screen until timing T11 atwhich the subject is designated, and the tracking icon 505 is displayedon the screen when the subject is designated as illustrated in FIG. 15B.Subsequently, even when the designated subject 503 moves to bepositioned away from the center of the image, tracking control isperformed based on the tracking correction amount calculated by thetracking amount calculation unit 118 such that the subject 503 returnsto the center of the image as illustrated in FIG. 15B. In the drawings,T15 indicates the timing at which a subject is designated and thesubject position is detected and T16 indicates the timing at which thesubject returns to the center of the image by the tracking control.

Subsequently, as illustrated in FIG. 15C, although tracking control isperformed while further increasing the tracking correction amount whenthe subject moves further, as illustrated in FIG. 15D, when the trackingtarget subject moves behind another subject and disappears from thescreen, the subject detection unit 117 (the subject position detectionunit 1000) cannot detect the subject and the subject lost state iscreated (T17). The subject lost state is created for a predeterminedperiod (T18) after the subject is lost, and the state can return to thesubject detection state when the subject 503 is detected again in thesubject lost state. When the predetermined period T18 has elapsed afterthe state transitions to the subject lost state, it is determined thatthe subject cannot be detected further and a subject detection stopstate is created. When the subject detection stop state is created, thestate does not transition to the subject detection state even when thesubject 503 appears in the screen again. When the subject detectionstate is to be created in the subject detection stop state, the subjectis designated again. Since there is a possibility to detect the subjectagain in the subject lost state, subject tracking control is continuedfor a predetermined period (T19) after the subject is lost. The subjecttracking control is stopped after the predetermined period T19 iselapsed, and a last tracking correction amount obtained based on a lastsampling data is held as the tracking correction amount which is theoutput of the tracking amount calculation unit 118. During the perioduntil T18 has finished, the last tracking correction amount ismaintained as the tracking correction amount, and when T18 has finished,the tracking control amount gradually approaches 0, and the correctionlens is returned to a position at which the subject tracking control isnot performed.

In the subject lost state (T18) , the tracking icon 505 is displayed ingray to notify the photographer of the fact that tracking control cannotbe performed. Moreover, when the period T18 has elapsed from the subjectlost state and the subject detection is stopped, the display of thetracking icon 505 is stopped.

In this manner, the CPU 105 functioning as the tracking statedetermination unit 120 determines that the tracking control cannot beperformed according to the subject detection state. Moreover, the CPU105 also functions as the tracking icon control unit 123 changes thedisplay of the tracking icon 505 (including stopping the display of thetracking icon) such that it is possible to urge the photographer toframe the camera to move the subject toward the center of the image andto set the tracking target subject again.

Next, an example of a method of sending a notification to thephotographer by changing the tracking icon 505 when tracking controlcannot be performed in a state in which the magnitude of the shake ofthe camera 101 exceeds a magnitude that can be corrected by image blurcorrection will be described with reference to FIGS. 16A to 16H. Whenthe detected shake amount of the camera 101 is large, it is determinedthat the tracking control cannot be performed. In this case, an icon isdisplayed to indicate that tracking control cannot be performed.

Similarly to the method described in FIG. 12A, when a subject isdesignated by a touch operation in FIG. 16A, tracking control starts andthe tracking icon 505 is displayed on the screen.

FIG. 16B is a diagram illustrating a state after the subject isdesignated. FIG. 16E illustrates the output of the adder 115 obtained byadding the blur correction amount and the tracking correction amountwhen the state of the screen displayed by the imaging apparatus 101changes from FIG. 16A to FIG. 16B and FIG. 16C according to the elapseof time. Moreover, FIG. 16E illustrates a center shift amount, FIG. 16Fillustrates the output of the angular velocity meter 103, FIG. 16Gillustrates a blur correction angle which is the output of the blurcorrection angle calculation unit 109, and FIG. 16H illustrates adisplay state of the tracking icon 505.

The tracking icon 505 is not displayed on the screen until timing T11 atwhich the subject is designated, and the tracking icon 505 is displayedon the screen when the subject is designated as illustrated in FIG. 16B.Subsequently, even when the designated subject 503 is away from thecenter of the image, tracking control is performed based on the trackingcorrection amount calculated by the tracking amount calculation unit 118such that the subject 503 returns to the center of the image asillustrated in FIG. 16B. In the drawings, T22 indicates the timing atwhich a subject is designated and the subject position is detected andT22 indicates the timing at which the subject returns to the center ofthe image by the tracking control.

Subsequently, when the shake amount of the imaging apparatus 101increases, the angular velocity which is the output of the angularvelocity meter 103 increases, and the blur correction angle calculatedbased on the output of the angular velocity meter also increases. Inthis case, when the blur correction angle exceeds a blur correctionlimit, the photographed image blurs and it is not possible to fill thesubject 503 into a certain position as illustrated in FIG. 16C. Thecorrection lens 108 cannot perform correction over a correction limit(Th5,−Th5). Thus, when the tracking state determination unit 120determines that tracking control cannot be performed in the rangebetween Th5 and −Th5 at T24, the tracking icon 505 is displayed in grayto notify the photographer of the fact that tracking control cannot beperformed.

As a method of determining whether tracking cannot be performed sincethe shake amount is large, a method of making the determination based onthe angular velocity output by the angular velocity meter 103 may beused. The tracking state determination unit 120 determines that trackingcannot be performed if a period in which the correction amount exceeds athreshold (Th6 or more or −Th6 or smaller) within a predetermined periodT20 is equal to or larger than a predetermined period. The determinationmay be performed based on the number of times rather than the period. Inthis case, it is determined that tracking cannot be performed when thenumber of times the correction amount, detected at a predeterminedsampling interval exceeds the threshold is equal to or larger than apredetermined number of times.

As another method of determining whether tracking cannot be performedsince the shake amount is large, a method of making the determinationbased on the blur correction angle which is the output of the blurcorrection angle calculation unit 109 may be used. In this case, thetracking state determination unit 120 determines that tracking cannot beperformed if a period or the number of times in which the correctionamount exceeds a predetermined threshold (Th7 or more or −Th7 orsmaller) within a predetermined period T21 is equal to or larger than apredetermined period or a predetermined number of times.

According to any one of these methods, it is possible to detect a statein which a large image blur continuously appears and it is difficult toperform image blur correction and tracking control. Moreover, it ispossible to change the display of the tracking icon 505 to notify thephotographer of the fact that tracking control cannot be performed.

In this manner, when a shake having a magnitude that cannot be correctedby the blur correction of the imaging apparatus occurs, the CPU 105functioning as the tracking state determination unit 120 determines thattracking control cannot be performed. Moreover, the CPU 105 alsofunctions as the tracking icon control unit 123 and changes the displayof the tracking icon 505 in response to the determination result of thetracking state determination unit 120 such that it is possible to notifythe photographer of the fact that the photographer needs to reliablyhold the imaging apparatus so that the shake of the camera 101decreases.

As described above, in the present embodiment, the tracking statedetermination unit 120 determines whether tracking control can beperformed. When it is determined that tracking cannot be performed, awarning is sent to the photographer such that it is possible to urge thephotographer to frame the imaging apparatus to move the subject towardthe center of the image and to reliably hold the imaging apparatus.

In the present embodiment, although a method of sending a warningnotification using the display of the tracking icon 505 has beendescribed, the following methods may be used as another method ofsending a warning.

(1) A warning notification is sent using the tracking icon 505.Specifically, a notification to the photographer indicating thattracking control cannot be performed is realized by blinking the icon onand off. The state in which tracking control cannot be performed may beclassified into several levels, and the photographer may be notified ofthe level by changing the blinking period according to the level.

(2) As illustrated in FIG. 17, an LED 1101 (a light emitting unit) maybe provided in the camera 101 and a warning notification is sent usingthe LED 1101. Specifically, a notification to the photographerindicating that tracking control cannot be performed is realized byblinking the LED 1101 on and off. The state in which tracking controlcannot be performed may be classified into several levels, and thephotographer may be notified of a level of the state in which trackingcontrol cannot be performed by changing the blinking period according tothe level. Moreover, the LED 1101 may be turned on when tracking can beperformed and the LED 1101 may be turned off when a subject is notdesignated and tracking control is not performed.

(3) As illustrated in FIG. 18, a sheet-shaped actuator 1201 thatgenerates vibration is provided at a position at which the photographerof the camera 101 usually places his or her finger when the photographergrasps the camera. A notification to the photographer indicating thattracking control cannot be performed is realized by vibrating theactuator 1201. A piezoelectric actuator which uses a piezoelectricelement, for example, is used as the actuator 1201. The state in whichtracking control cannot be performed may be classified into severallevels, and the photographer may be notified of the level of the statein which tracking control cannot be performed may be notified to thephotographer by changing the magnitude of the vibration according to thelevel.

In FIGS. 12A to 12H to FIGS. 16A to 16H, an example in which it isdetermined whether tracking control can be performed using onedetermination method has been described. However, it is determinedwhether tracking control can be performed using a plurality ofdetermination methods. For example, it may be determined that trackingcontrol cannot be performed when a state in which the correction amountis equal to or larger than the trackable threshold Th1 continues thepredetermined period TL or longer, and it may be determined thattracking control cannot be performed when a state in which a positionshift amount is equal to or larger than the threshold Th3 continues thepredetermined period T7 or longer. The determination methods may becombined appropriately. In this case, warning display may be changedaccording to the determination method, and warning display may bechanged depending on the reason why tracking control cannot be performed(due to a large correction amount, due to the subject detection state,or due to a large shake). However, since it may appear complicated tothe photographer if there are a number of types of warning display, itis preferable to use the same warning display regardless of adetermination method and the reason why tracking control cannot beperformed. For example, it is preferable to use the same warning displayregardless of whether the reason is due to a large correction amount ora subject detection state. This is because even if there is one warningdisplay, it is possible to promote the operation in which thephotographer tracks the subject by a framing operation by using thewarning notification.

In the present embodiment, a so-called optical shift method of moving acorrection lens within a plane vertical to an optical axis of aphotographic optical system is used as an automatic subject trackingunit. However, the present invention is not limited to the optical shiftmethod but the following configurations may be employed.

(1) A configuration for moving an imaging element within a planevertical to an optical axis.

(2) A configuration for changing a slicing position of respectivephotographic frames output by an imaging element.

(3) A configuration for rotating a lens barrel including an imagingelement and a photographing lens group.

(4) A configuration combined with a rotary pan head capable of panningand tilting an imaging apparatus, provided separately from the imagingapparatus.

(5) A configuration in which the plurality of configurations arecombined.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment (s) , and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment (s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU) , micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM) , a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as acompact disc (CD) , digital versatile disc (DVD) , or Blu-ray Disc(BD)™) , a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-244288, filed Dec. 15 2015, Japanese Patent Application No.2015-239747, filed Dec. 8 2015, and Japanese Patent Application No.2016-218297, filed Nov. 8 2016, which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. A control device comprising at least oneprocessor or one circuit which functions as: a correction control unitconfigured to acquire a blur detection signal detected by a blurdetection unit to obtain a correction amount of an image blur andcontrol an image blur correction unit configured to correct the imageblur; a subject detection unit configured to detect a position of asubject in a photographed image and acquire position information of thesubject in the photographed image; and a setting unit configured to setasubject selectable mode in which a user is able to select a desiredsubject, wherein an image blur correction effect in a second state inwhich the subject selectable mode is set and the desired subject isselected is higher than an image blur correction effect in a first statein which the subject selectable mode is set and the desired subject isnot selected.
 2. The control device according to claim 1, the processoror the circuit further function as: a tracking control unit configuredto perform tracking control of the desired subject selected in thesubject selectable mode based on the position information of the desiredsubject acquired by the subject detection unit.
 3. The control deviceaccording to claim 2, wherein the correction control unit changes thedegree of an image blur correction effect based on a subject detectionresult obtained by the subject detection unit.
 4. The control deviceaccording to claim 2, wherein the tracking control unit performstracking control of the desired subject selected in the subjectselectable mode by moving the position of the desired subject based onthe position information of the desired subject acquired by the subjectdetection unit.
 5. The control device according to claim 1, wherein thecorrection control unit includes a filter for calculating the correctionamount and changes the degree of the image blur correction effect bychanging characteristics of the filter.
 6. The control device accordingto claim 5, wherein the characteristic of the filter is the cutofffrequency of the filter.
 7. The control device according to claim 6,wherein the correction control unit raises the image blur correctioneffect by lowering the cutoff frequency of the filter used in the firststate more than the cutoff frequency of the filter that is used in thesecond state.
 8. The control device according to claim 2, wherein thecorrection control unit acquires the blur detection signal to determinewhether panning or tilting is performed, changes the characteristics forcalculating the correction amount according to a determination result,and performs control to change the degree of the image blur correctioneffect by performing a process of shifting the correction amount furthercloser to a center of a control range when it is determined that panningor tilting is performed than that when it is determined that panning ortilting is not performed.
 9. The control device according to claim 4,wherein the tracking control unit performs the tracking control of thesubject by moving the position of the subject in the photographed imageto decrease the distance between the position of the subject in thephotographed image and a predetermined position in the photographedimage, and wherein the correction control unit performs control todecrease the image blur correction effect when the distance between theposition of the subject and the predetermined position is equal to orlarger than a threshold to be lower than the image blur correctioneffect when the distance between the position of the subject and thepredetermined position is smaller than the threshold.
 10. The controldevice according to claim 1, the processor or the circuit furtherfunctions as: a reliability determination unit configured to compare areliability of the detected subject with a threshold to determine thereliability of the subject, wherein the correction control unit performscontrol to increase the image blur correction effect when thereliability is equal to or larger than the threshold to be higher thanthe image blur correction effect when the reliability is smaller thanthe threshold based on a determination result obtained by thereliability determination unit.
 11. The control device according toclaim 1, wherein the correction control unit includes a filter forcalculating the correction amount and changes the degree of the imageblur correction effect by changing the characteristics of the filter.12. The control device according to claim 1, the processor or thecircuit further functions as: a determination unit configured todetermine whether tracking control of the subject is able to beperformed; and a warning instruction unit configured to issue a warningwhen the determination unit determines that tracking control of thesubject is not able to be performed.